Sample complexity of stochastic least squares system identi � cation

In this paper we consider the nite sample properties of least squares identi cation in a stochastic framework The problem we pose is How many data points are required to guarantee with high probability that the expected value of the quadratic identi cation criterion is close to its empirical mean value The sample sizes are obtained using risk minimisation theory which provides uniform probabilistic bounds on the di erence between the expected value of the squared prediction error and its empirical mean evaluated on a nite number of data points The bounds are very general No assumption is made about the true system belonging to the model class and the noise sequence is not assumed to be uniformly bounded Further analysis shows that in order to maintain a given bound on the deviation the number of data points needed grows no faster than quadratically with the number of parameters for FIR and ARX models Results for general asymptotic stable linear models are also obtained by considering ARX approximations of these models The sample sizes derived for FIR and ARX models di er dramatically from the exponential sample sizes obtained for deterministic worst case l identi cation However the setting and problem considered in these two cases stochastic least squares and deterministic l are completely di erent so the sample sizes are not really comparable